Improving video streaming over WAN

BBRv3 vastly improves reliability of video streaming over a high latency link.

CachyOS is the only Linux distro that has this builtin to its kernel. It must be enabled manually.

This ansible config provides a way to set this up on new CachyOS and Debian machines. It includes a kernel build for Debian 12.

Some sysctls are needed to increase TCP buffer sizes. See also this thread:

# BBRv3
net.ipv4.tcp_congestion_control = bbr
net.core.default_qdisc = fq
# Needed for cloudflared quic:
# https://github.com/cloudflare/cloudflared/issues/1176
# It may also be needed for BBR:
# https://groups.google.com/g/bbr-dev/c/Yj2_WCBtXpQ/m/S2976NUzEQAJ
# Note: Setting _default settings affects non-IP sockets as well.
net.core.rmem_max=900000000
net.core.wmem_max=900000000
net.ipv4.tcp_wmem=4096 262144 900000000
net.ipv4.tcp_rmem=4096 540000 900000000

iperf3 examples

Without BBRv3 (default Debian 12 kernel)

[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  1.15 MBytes  9.67 Mbits/sec
[  5]   1.00-2.00   sec  2.52 MBytes  21.1 Mbits/sec
[  5]   2.00-3.00   sec  1.78 MBytes  15.0 Mbits/sec
[  5]   3.00-4.00   sec  2.30 MBytes  19.3 Mbits/sec
[  5]   4.00-5.00   sec  1.24 MBytes  10.4 Mbits/sec
[  5]   5.00-6.00   sec  1.24 MBytes  10.4 Mbits/sec
[  5]   6.00-7.00   sec  1.55 MBytes  13.0 Mbits/sec
[  5]   7.00-8.00   sec  1.36 MBytes  11.4 Mbits/sec
[  5]   8.00-9.00   sec  1.37 MBytes  11.5 Mbits/sec
[  5]   9.00-10.00  sec  1.58 MBytes  13.3 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.17  sec  19.1 MBytes  15.7 Mbits/sec  288             sender
[  5]   0.00-10.00  sec  16.1 MBytes  13.5 Mbits/sec                  receiver

With BBRv3

me@client:~$ iperf3 -c my.server.org -R
Reverse mode, remote host my.server.org is sending
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   764 KBytes  6.26 Mbits/sec
[  5]   1.00-2.00   sec  14.3 MBytes   120 Mbits/sec
[  5]   2.00-3.00   sec  13.7 MBytes   115 Mbits/sec
[  5]   3.00-4.00   sec  10.1 MBytes  85.0 Mbits/sec
[  5]   4.00-5.00   sec  12.2 MBytes   103 Mbits/sec
[  5]   5.00-6.00   sec  11.1 MBytes  93.0 Mbits/sec
[  5]   6.00-7.00   sec  12.4 MBytes   104 Mbits/sec
[  5]   7.00-8.00   sec  11.7 MBytes  98.1 Mbits/sec
[  5]   8.00-9.00   sec  13.6 MBytes   114 Mbits/sec
[  5]   9.00-10.00  sec  13.7 MBytes   115 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.16  sec   117 MBytes  96.3 Mbits/sec  900             sender
[  5]   0.00-10.00  sec   114 MBytes  95.3 Mbits/sec                  receiver